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An  Automated  Low  Cloud  Prediction  System 


I.  INTRODUCTION 

The  USAF  Air  Weather  Service  (AWS)  recognizes  ihe  need  to  modernize  its 
weather  support  capabilities.  The  Automated  Weather  Distribution  Svstem  (AU'DS) 
Multi-Command  Required  Operational  Capability  (ROC-801-77)  and  the  AWS  GR 
3-76  (Automated  Short-Range  Terminal  Forecasts)  identify  the  need  to  provide  re¬ 
sponsive  environmental  support  to  local  decision  makers.  This  support,  in  the 
form  of  short-range  forecasts,  must  provide  a  statistically  significant  improve¬ 
ment  over  decision  assistance  provided  bv  persistence.  In  response  to  these  USAF 
requirements,  the  Air  Force  Geophysics  Laboratory  (AFGL)  has  developed  lore- 

cast  schemes  which  provide  probability  estimates  for  several  critical  weather 

1  2  3 

elements:  radiation  fog,  slant  visual  range,  and  visibility. 

The  objective  of  this  study  is  to  develop  automated  procedures  for  probability 
and  categorical  forecasts  of  low  cloud  ceiling,  defined  here  to  be  less  than  6000  ft. 
The  approach  relies  upon  the  automated  cloud  observing  system  developed  bv  AFGL 


(Received  for  publication  22  dune  1981) 

1.  Tahnk,  W.  R.  (1975)  Objective  Prediction  of  Fine  Scale  Variations  in  Radiation 

Fog  Intensity,  AFC'FTE^TR  -7T>  -025F,  AD 'AO  1477‘4. - 

2.  Geisler,  E.  B.  (1979)  Development  and  Evaluation  of  a  Tower  Slant  Visual  Range 

System.  AEG L-TR -7 9-02097  AD  A082384. 

3.  Hering,  W.  S.  ,  and  Quick,  W.  L.  ( 1974)  Hanscom  visibility  forecasting  experi  - 

ment,  Proc.  of  5th  Conference  on  Weather  Forecasting  and  Analysis,  pp  224-227. 
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to  provide  the  necessary  data  bases  to  generate  and  verify  forecast  equations. 
Climatology,  persistence.  Equivalent  Markov,  and  regression  estimation  of  event 
probabilities  (KEEP)  prediction  algorithms  were  applied  to  generate  probability 
forecasts.  Categorical  forecasts  were  retrieved  from  the  probability  forecasts  by 
four  techniques:  maximum  probability,  persistence,  unit  bias,  and  iterative.  The 
probability  and  categorical  forecast  techniques  were  evaluated  and  comparisons 
made  to  determine  their  respective  accuracy  and  reliability,  forecast  lengths 
examined  in  this  study  are  30,  60,  120,  and  180  ntin. 

2.  PROBABILITY  FORECAST  TECHNIQUES 

2. 1  Regression  Estimation  of  Event  Probabilities  (REEP) 

4 

REEP  is  analogous  to  a  forward  screening  regression  technique  where  a  sub¬ 
set  of  predictors  is  selected  from  a  large  set  of  possible  predictors.  The  relation¬ 
ship  between  the  predictand  and  the  selected  predictors  is  a  linear  function  whose 
coefficients  are  determined  by  least  squares.  REEP  calculates  probabilities  of 
being  within  categories.  These  within-category  probabilities  can  be  easily  con¬ 
verted  into  exceedance  probabilities.  REEP  has  been  successfully  used  in  other 
forecasting  experiments  conducted  at  AFGL  by  Tahnk1  and  Geisler.  ^ 

REEP  requires  each  raw  variable  be  broken  down  into  a  set  of  two  or  more 
mutually  exclusive  and  exhaustive  categories  called  dummy  variables.  The  dummy 
variable  categories  are  assigned  a  value  of  1  if  the  continuous  variable  falls  within 
the  range  of  the  category  limits,  or  0  if  it  is  outside  the  range.  Therefore,  lor  any 
one  raw  variable  broken  down  into  n  dummy  variables,  one  dummy  variable  will  be 
assigned  a  value  of  1  and  all  of  the  others  (n-l>  will  be  assigned  a  value  of  0. 

The  result  of  the  least  squares  technique  is  a  set  of  equations,  one  equation 
for  each  predictand  category  (i). 
k 

P.  =  An  +  1  A.  X.,  i  -  1,2 _ 5  (1) 

i  0.  j=i  ij  J 

where  k  is  the  number  of  dummy  predictors  (XT  selected  by  a  screening  process, 
and  the  A's  the  regression  constants  and  coefficients. 

REEP  is  formulated  to  insure  internal  consistence  among  predictand  categories 
such  that  the  sum  of  probabilities  totals  unity.  This  is  insured  by  using  the  same 
predictor  categories  for  each  predictand  equation.  Each  predictand  equation  yields 
a  probability  of  occurrence  (Pj)  for  "its"  category. 

4.  Miller,  R.G.  (1964)  Regression  Estimation  of  Event  Probabilities,  Tech.  Rpt. 
7411-121,  The  Travelers  Research  Center,  Inc.,  Hartford,  Connecticut 
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2.2  Equivalent  Markov  Baaed  on  REEP 
5 

Miller  proposed  a  prediction  method  yielding  probabilistic  forecasts  compar¬ 
able  to  the  classical  Markov  process  but  without  the  necessity  of  utilizing  the 

6  2 

Markov  transition  matrix  M  explicitly.  Whiton  and  Geisler  found  this  technique 
to  yield  results  comparable  to  the  classical  Markov  process  but  much  easier  to 
develop  and  apply  to  practical  forecasting  problems. 

The  KEEP  equations  provide  a  conditional  probability  matrix  P  equivalent  to 
M.  One  assumes  the  categories  (predictor  and  predictand)  used  in  REEP  form  a 
finite  Markov  chain  (XT.  i  1,2,..,  5.  Hence,  P  is  a  square  matrix. 

The  stochastic  process  ( Xj )  has  the  Markovian  property 

P(Xt+1--i|  Xt=.i)=P.  ;  i.  j=  1,  2.  . ,  5  .  (2) 

The  existence  of  one-step  transition  probabilities  P.^  implies  that  for  each 
i,  j  and  n 

P<Xt+n  =  ilxt=j)=P(Xn=i|Xo=j>  -  Pij(n).  (3) 

P is  the  conditional  probability  the  random  variable  X,  starting  in  state  i,  will 
be  in  state  i  after  exactly  n  steps  (time  units)  and  is  called  the  n-step  transition 
probability. 

More  generally,  the  matrix  of  n-step  transition  probabilities  can  be  obtained 
from  the  expression: 

P(n)  P.  P.  .  .  P  =  Pn  P-p"'1  P0'1  P  .  (4) 

Thus,  the  n-step  transition  probability  matrix  can  be  determined  by  computing  the 
nth  power  of  the  one-step  transition  matrix  P.  In  this  application  of  the  Equivalent 
Markov  model,  a  30-min  one-step  transition  matrix  was  used  to  generate  predic¬ 
tions  from  30  to  180  min  (see  Appendix  B).  For  example,  to  generate  the  120-min 
forecast  probabilities,  one  must  first  raise  P  to  a  power  of  4  and  then  pre -multiplv 
the  resultant  transition  matrix  by  O,  the  observation  matrix: 


o  =  (Oj.  o2. 


5.  Miller,  R.G.  (1968)  A  stochastic  model  for  real-time  on-demand  weather  pre¬ 

dictions,  Proe.  1st  Statistical  Meteorological  Conference,  Am.  Meteor.  Soc. 
pp  48-51.  -  “ 

6.  Whiton,  R.W.  ( 1977  )  Selected  Topics  in  Statistical  Meteorology,  AWS-TR-77  - 

273,  pp  7-1  to  7 -45T  ' 


where 


k  predictor  category  number 

zero,  if  category  is  not  observed 
k 

one,  if  category  is  observed 


2.3  Climatology 

Climatology  forecasts  were  computed  using  the  Wind  Stratified  Conditional 
Climatology  (U  SCC )  and  Kevised  Uniform  Summary  of  Surface  H  eat  her  Observations 
(HUSSWO)  climatology  tables.  These  tables  were  based  on  approximately  20  years  o> 
hourly  human  observations  taken  at  Otis  A  KB,  located  >n  Cape  Cod,  .Massachusetts. 

These  tables  have  common  characteristics  but  have  some  significant  differ¬ 
ences.  Both  tables  are  stratified  so  one  can  quickly  retrieve  the  frequency  of 
occurrence  for  any  event  (for  example,  ceiling  less  than  200  ft),  conditioned  on 
month  and  time  of  day.  However,  the  VVSCC  table  is  further  divided  to  condition 
the  event  on  wind  direction  and  cloud  ceiling.  This  allows  one  to  retrieve  iron,  die 
WSC'C  table,  forecasts  for  the  frequency  of  occurrence  of  •  1« .  oiling  for  periods 
of  1,  2,  and  3  h.  This  is  unlike  RLSSUO  which  provides  only  climatological  fre¬ 
quencies  for  a  particular  lime  of  day  and  month. 

In  our  use  of  the  VVSCC  table,  we  c  hose  to  condition  the  modi,  land  event,  noun 
c  eiling,  on  three  variables:  month,  time  of  day,  and  initial  loud  .  oiling  category. 

If  we  had  elected  to  also  condition  the  event  on  wind  direction,  some  -if  die  trequenev 
of  occurrence  values  obtained  from  the  VVSCC  table*  would  have  hem  statistically 
unstable,  because  they  were  based  upon  too  few  observations. 

In  generating  climatologic  al  forecasts  only  one  table  was  used  tor  each  tore- 
cast  made.  If  a  ceiling  condition  was  reported,  the  VVSCC  table  was  used.  A 
30-niin  climatology  forecast  was  not  made  because  the  VVSCC  table*  onlv  provides 
hourly  forecasts.  If  a  no  ceiling  condition  was  observed,  then  tile  RKSSWO  table 
was  used. 


2.4  Persistence 

The  model  used  as  a  control  technique  in  this  experiment  was  pers  ist<*ncc. 
Here,  one  takes  the  initial  c  loud  ceiling  category,  assigns  a  probability  of  1.  0  to 
it  and  0.0  to  the  remaining  categories  and  assumes  that  conditions  will  not  change 
during  the  period  of  the  forecast. 
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3.  INSTRUMENTATION  AND  DATA  SETS 
3.1  Instrument  Configuration 

Measurements  of  cloud  base  height  (AN/GMQ-13  Rotating  Beam  Ceilometer: 
RBC),  wind  direction  (Climatronics  Mark  I  Wind  Sensor),  and  visibility  (EG&G  Model) 
207  Forward  Scatter  Meter:  FSM)  were  obtained  at  the  AFGL  Weather  Test  Facility 
(WTF)  located  at  Otis  AFB,  Massachusetts.  The  FSM  and  Climatronics  Wind 

7 

Sensor  have  been  used  in  several  experiments  carried  out  by  AFGL  during  the  past 
several  years.  Figure  1  shows  the  configuration  of  the  surface-based  instruments 
used  in  the  study. 


FORWARD  SCATTER 
METER 


ROTATING  BEAM 
CEILOMETER 


WIND  sc.  q,  [ 


TRANSMITTER 


Figure  1.  Instrument  Configuration 


The  RBC  was  used  to  acquire  cloud  height  data  in  these  tests.  The  RBC  is 
the  standard  cloud  height  measuring  device  used  by  the  L’SAF  Air  Weather  Service, 
the  National  Weather  Service,  and  internationally.  It  consists  of  a  two-lamp  pro¬ 
jector  system  and  a  receiver  which  is  normally  set  400  ft  from  the  projector  with 
its  field  of  view  vertical  and  coplanar  with  the  rotating  projector  beams.  The  inter¬ 
section  volume  moves  upward  as  each  projector  beam  scans  to  the  vertical.  When 
the  volume  intersects  a  cloud,  backscatter  of  the  projector  beam  by  particles  in  the 
cloud  is  detected  by  the  receiver  and  displayed  on  the  indicator  as  a  height  vs  in¬ 
tensity  depiction.  The  projector  angle  at  which  the  maximum  backscatter  return 
occurs  represents  the  cloud  height.  Cloud  heights  in  our  tests  were  limited  to 
measurements  up  to  6000  ft  because  of  known  sensor  and  trigonometric  limitations 
and  after  an  assessment  of  basic  RBC  data. 

7.  Chisholm,  I). A.,  Lynch,  R.  H.  ,  Weyman,  .I.C.,  and  Geisler,  E.  B.  (1980)  A 

Demonstration  of  the  Modular  Automated  Weather  System  (MAWS). 

A FG I . -tR -80-0087 ,  At“ A0S7070.  ' 

8.  Weyman,  J.C.  ,  and  Lynch,  R.  H.  (1981)  A  Digital  Processing  and  Display  System 

to r  the  Rotating  Beam  C'eilom etc r  (AN  /GMQ-  13  ),  A  FG  L-TR  -8  1  -  6621 . 


Measurements  of  visibility  were  obtained  from  the  FSM  in  extinction  coefficient 
units.  Data  were  collected  at  a  rate  of  5  observations  per  min  and  were  subse¬ 
quently  processed  to  yield  a  1-min  average  of  extinction  coefficient.  One  can  use 
Koschmeider 's  Law  (for  daytime  conditions)  and  Allard's  Law  (for  nighttime  con¬ 
ditions)  to  explicitly  relate  visibility  to  extinction  coefficient.  Figure  2  shows 


representative  values  of  day  and  night  visibility  (in  meters)  for  a  given  value  of 
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extinction  coefficient  (in  units  of  km  ). 


Figure  2.  Relation  of  Extinction  Coefficient 
to  Visibility 


The  Climatronics  cup-and-vane  wind  sensor  is  lightweight,  has  low  power  con¬ 
sumption  and  a  low  start-up  threshold.  (A  wind  speed  of  0.  22  m/sec  turns  the  cups 
and  0.  11  m/sec  moves  the  wind  direction  vane.  )  It  also  responds  quickly  and  is 
very  accurate  (for  wind  speed,  ±  1  percent  or  ±0.7  m/sec,  whichever  is  greater, 
and  for  direction  ±2.5“).  A  non -contacting  wind-direction  transducer  and  a 
chopped  solid-state  light  source  sense  direction  and  speed.  Measurements  were 
collected  at  a  rate  of  1  observation  per  sec  and  processed  to  yield  a  1-min  average 
of  wind  direction  and  speed. 


3.2  Data  Set* 


Data  wort*  collected  from  March  to  Dcci'mlicr  1980.  Wind  direction,  visibility 

and  cloud  bast'  height  measurements  were  placed  on  magnetic  tape  and  subsequently 

processed  on  the  CDC  6600  system.  This  processing  transformed  the  raw  cloud 

base  height  values  into  an  objectively  determined  low  cloud  observation  bv  applying 

the  hierarchical  clustering  technique.  The  hierarchical  clustering  technique  has 
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been  shown  to  yield  stable,  reliable,  and  representative  cloud  observations.  ’ 

It  is  these  automated  cloud  observations  which  made  up  the  population  from  which 
episodes  were  selected  for  this  study.  In  all,  43  episodes  were  chosen.  Episodes 
were  selected  if  they  satisfied  three  criteria:  (1)  episode  duration  was  at  least 
3  h,  (2)  clouds  were  reported  below  6000  ft,  and  (3)  a  ceiling  was  reported  at 
some  time  during  the  episode.  Six  of  the  episodes,  containing  approximately  7  000 
observations,  were  randomly  chosen  and  set  aside  as  an  independent  data  set.  The 
r  emaining  37  episodes,  comprised  of  approximately  20,000  observat ions,  made  up 
the  depend°nt  data  set. 

Four  predietand  categories  were  selected  o  coincide  with  the  first  four  cloud 
ceiling  categories  of  the  WSCC  and  the  remaining  events  in  the  sample  comprised 
a  fifth  category.  Table  1  lists  the  category  limits. 


Table  1.  Predietand  Category  Limits 


Predietand  Category 

Limits  (ft) 

1 

Ceiling  <  200 

2 

200  -  Ceiling  <  500 

3 

500  ?  Ceiling  <.  1000 

4 

1000  -  Ceiling  <  3000 

3 

3000  ?  Ceiling  <-  6000 

or 

No  Ceiling 

Table  2  summarizes  the  relative  frequency  distribution  for  the  developmental 
and  independent  data  sets.  Note  that  the  developmental  data  frequency  distribution 
represents  a  randomly  selected  sub-sample  taken  from  the  dependent  data  set. 

The  limit  on  sample  size  of  3000  is  imposed  by  the  REEP  computer  programs. 


9.  Geisler,  E.  B.  ,  and  Chisholm,  D.  A.  (1980)  An  Automated  Cloud  Observing 
System  (ACOS)  AFGL-TR-81-002,  AD  A 100266. 

10.  Duda,  R.O.  ,  Mancuso,  R.  L.  ,  and  Paskert,  P.  F.  (197  1)  Analysis  of  Techniques 
for  Describing  the  State  of  the  Sky  Through  Automation,  Report  No.  F>AA»Rb- 
7  1-52,  prepared  for  Department  of  Transportation,  Washington,  D.  C. 
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Table  2.  Frequency  Distribution  of  Independent  and 
Developmental  Data  by  Fredictand  Categories 


Fredictand  Category 

Developmental 

Observations 

Independent 

Observations 

I 

145 

120 

2 

547 

1211 

3 

917 

2276 

4 

674 

1547 

5 

7  17 

187  3 

Total 

3000 

7027 

Three  raw  variables  were  selected  to  constitute  the  predictor  set:  Low  cloud 
observation,  visibility,  and  wind  direction.  Category  limits  were  assigned  to 
visibility  and  wind  direction  based  on  their  relative  frequency  distributions  in  the 
dependent  data  set.  The  resultant  categories  for  the  predictors  are  listed  in 
Table  3.  The  predictor  categories  were  used  as  input  to  each  of  the  probability 
prediction  techniques. 


Table  3.  Predictor  Categories 


Predictor  Category 

Type  (Units) 

!  .imits 

1 

Cloud  Height  (ft) 

Ceiling 

< 

200 

2 

Cloud  Height  (ft) 

200  <  Ceiling 

< 

500 

3 

Cloud  Height  (ft) 

500  5  Ceiling 

< 

1000 

4 

Cloud  Height  (ft) 

1000  <  Ceiling 

< 

3000 

5 

Cloud  Height  (ft) 

3000  f  Ceiling 

< 

6000 

6 

Extinction  Coefficient 

EC  (km  * ) 

EC  (km  S 

Or  No  Ceiling 

EC  ? 

0.  4 

7 

0.  4  <  EC 

< 

2.0 

8 

EC  (km'1) 

2.  0  <  EC 

- 

25.  0 

9 

10 

EC  (km  *) 

Wind  Direction  (deg) 

25. 0<  EC 

WD  5  320  or  WD  <  100 

11 

Wind  Direction  (deg) 

100  <  WD 

< 

160 

12 

Wind  Direction  (deg) 

160  <  WD 

< 

230 

13 

Wind  Direction  (deg) 

230  <  WD 

< 

320 

Three  variations  of  possible  predictors  were  evaluated  for  KEEP.  This  allowed 
a  determination  of  the  additive  contribution  of  visibility  and  wind  direction  informa¬ 
tion  to  prediction  accuracy  beyond  that  attained  using  only  the  initial  cloud  observa¬ 
tion  as  a  predictor.  Hence,  three  KEEP  equation  sets  were  generated  from  a 
randomly  selected  sub-sample  of  3000  observations  taken  from  the  dependent  data 
set.  The  first  set,  KEEP5,  used  only  the  initial  cloud  observation.  The  second 
set,  KEEP.  01,  used  selected  predictors  which  reduced  the  predictand  variance  by 
at  least  1  percent.  The  final  set,  KEEP11,  used  all  the  predictors  selected  by 
least  squares  with  no  restriction  on  the  amount  of  variance  reduced.  Appendix  A 
lists  the  KEEP  equation  sets. 

4.  CATEGORICAL  FORECAST  TECHNIQUES 

There  are  a  variety  of  methods  by  which  a  categorical  forecast  can  be  made. 

In  this  experiment,  we  chose  to  compare  four  methods:  maximum  probability, 
iterative,  unit  bias,  and  persistence.  The  first  three  methods  are  thresholding 
processes  which  transform  a  probability  forecast  into  a  categorical  forecast. 
NOAA/TDL**  has  used  thresholding  methods  with  excellent  results.  Since  the 
thresholding  approach  requires  probability  estimates  of  the  events  forecasted,  a 
KEEP  equation  set  was  generated  from  the  developmental  sample.  The  categories 
or  events  were  (1)  ceiling  less  than  1000  ft,  and  (2)  either  ceiling  greater  than  or 
equal  to  1000  ft  or  no  ceiling.  In  general,  if  the  KEEP  probability  exceeds  the 
threshold  probability  (p*)  for  the  first  category,  then  the  first  category  is  chosen 
to  be  the  categorical  forecast.  Otherwise,  the  second  category  is  selected.  The 
specific  methods  of  assigning  the  threshold  probability  are  discussed  below. 

Appendix  C  lists  the  KEEP  equation  set  used  here.  Categorical  forecasts  were 
made  for  all  four  forecast  intervals  (30,  60,  120,  and  180  min). 

4.1  Maximum  Probability 

The  maximum  probability  (Wax  Prob)  model  selects  the  category  with  the 
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largest  probability  as  the  categorical  forecast.  This  is  equivalent  to  a  threshold¬ 
ing  technique  with  p*  =  0.5.  It  is  perhaps  the  most  commonly  used  because  of  its 
simplicity.  REEP  and  climatology  probabilities  were  used  by  the  model.  KEEP 
(Max  Prob  R)  is  discussed  in  Section  2.  1  and  climatology  (Max  Prob  c)  in  Sec¬ 
tion  2.3.  The  use  of  both  methods  to  convertfrom  probability  to  categorical  forecasts 
enables  us  to  determine  any  difference  in  skill  between  them  in  making  the  conversion. 

11.  Miller,  R.  G.  ,  and  Best,  D.  L.  (1978)  A  model  for  converting  probability  fore¬ 

casts  to  categorical  forecasts,  NOAA  Tech.  Memo,  TDL.  Office  Note  78-14. 

12.  Miller,  R.  G.  ,  and  Best,  D.  L.  ( 19  7  9)  A  model  for  converting  probability  fore¬ 

casts  to  categorical  forecasts,  Preprints,  Sixth  Conf.  on  Probability  and 
Statistics  in  Atmospheric  Sciences,  Am.  Meteor.  Soc,  ,  pp  9(5-  1 02. ' 


4.2  Iterative 


The  iterative  technique  is  an  empirical  one.  On  successive  passes  through  the 
dependent  data  set,  the  descriptive  statistics  bias,  threat  score,  and  percent  cor¬ 
rect  (see  Section  5.  2)  were  computed  by  comparing  the  actual  probability  forecasts 
against  incremented  threshold  probabilities.  The  selected  threshold  probability 
is  then  subjectively  determined  based  mainly  on  the  bias  statistics.  Table  4  lists 
the  threshold  probabilities  selected  for  each  forecast  interval,  for  the  predictand 
category  ceiling  <  1000  feet. 


Table  4.  Threshold  Probabilities  for  Ceiling 
Less  Than  1000  ft  Selected  From  Iterative  Method 


Forecast  Length 

Threshold  Probability  of 

(min) 

Ceiling  <  1000  ft 

30 

0.  72 

60 

0.  69 

120 

0.  58 

180 

0.64 

4.3  Unit  Bias 

The  unit  bias  model  seeks  to  establish  thresholds  which  will  result  in  a  cate¬ 
gorical  bias  statistic  of  one  (unity)  for  the  developmental  sample.  Threshold  prob¬ 
abilities  are  calculated  by  the  unit  bias  model;  however,  they  are  determined  differ¬ 
ently  from  the  iterative  technique.  Here,  p*  is  a  function  of  the  climatology  (c)  of 
the  predictand  event,  ceiling  less  than  1000  ft,  and  the  multiple  correlation  coeffi¬ 
cient  (K)  between  the  predictand  and  predictors  in  the  forecast  equation.  This 
information  permits  a  fast  and  simple  calculation  of  p  ,  compared  to  the  iterative 
technique; 

p  :  =  K  (0.  5-c)  +  c  (6) 

Table  5  lists  the  (It,  c)  pairs  for  each  forecast  length. 

4.4  Persistence 

The  control  technique  was  persistence.  Whatever  the  initial  cloud  category 
was  at  time  zero,  the  categorical  forecast  was  to  keep  it  in  that  category  for  all 
forecast  lengths. 


Table  5.  Unit  Bias  Correlation  and 
Climatology  (H,  e)  Pairs 


Forecast 

Length 

(min) 

R 

c 

30 

0.  849 

0.  452 

60 

0.778 

0.455 

120 

0.  67  9 

0.  465 

180 

0.  594 

0.  466 

5.  EVALUATION  CRITERIA 


5. 1  Probability  Forecasts 


Pr  jibability  estimates  of  a  weather  event  must  be  accurate  and  reliable  to  be 
useful.  Over  a  period  of  time,  the  event  should  actually  occur  with  the  frequency 
represented  bv  the  probability  forecast.  In  addition,  resolution  in  the  probabilities 
is  desirable;  that  is,  they  should  be  as  close  to  0.  0  or  to  1.  0  as  possible  when  the 
event  does  not  occur  or  does  occur,  respectively. 

The  accuracy  of  the  probability  forecasts  was  determined  by  the  Brier  Prob¬ 
ability  (or  p)  Score which  measures  the  mean -squared  probability  errors.  It  is 
determined  from  h’q.  (7). 


P 


N  k 

I  E  E  <fh 

N  i=1  ri  'J 


O.  ) 


(7) 


where 


N 

P. 

i.l 

O. 

U 

k 


number  of  forecasts, 

probability  estimate  for  jth  predictand  category, 
observed  /alue  (0,  1)  for  ,|th  predictand  category, 
number  of  predictand  categories. 


The  range  ol  values  for  p  depends  on  the  number  of  predictand  categories.  For  two 
or  more  predictand  categories,  p  ranges  from  0.  0  to  2.  0.  For  one  predictand 
(ceiling  less  than  1000  ft)  p  ranges  from  0.  0  to  1.  0.  A  perfect  probability  forecast 
has  a  Brier  Score  of  0.0  and  the  worst  probability  forecast  has  the  maximum  value 
in  the  range. 

Reliability  of  the  probability  forecasts  can  be  depicted  by  a  simple  graphing 
technique  which  allows  us  to  compare  the  predicted  probability  distribution  to  the 
observed  frequency.  Figure  3  is  an  example  of  a  reliability  graph  in  which  eleven 


13.  Brier,  G.  IV.  (1050)  V  erification  of  forecasts  expressed  in  terms  of  probability, 
Mon.  Wea.  Rev.,  7  8:1-3. 
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probability  categories  and  observation  frequencies  are  used.  The  diagonal  line 
denotes  perfect  reliability  where,  for  example,  40  percent  of  the  cases  in  which 
the  probability  forecasts  ranged  from  0.  25  to  0.  45  resulted  in  the  forecasted  event 
actually  occurring.  The  bar  graph  to  the  right  depicts  the  proportion  of  probability 
forecasts  in  each  of  the  eleven  categories.  The  categories  used  are 

|(0,.  05),  (.05..  15).  (.15..  25), _ (.  85.  .  05).  (.05.  1.0)|. 

In  this  illustration  the  distribution  of  probability  forec  ast  reflects  fairly  poor 
resolution  because  the  majority  of  the  probabilities  are  "mid- range"  values 
(20-80  percent). 
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Figure  5.  Reliability  Graph  Fxample 


5.2  Categorical  Forecasts 

Categorical  forecasts  were  made  for  one  of  two  events:  (1)  ceiling  less  than 
1000  ft,  or,  (2)  ceiling  greater  or  equal  to  1000  ft,  which  includes  no  ceiling  cases. 

Categorical  forecast  skill  was  measured  using  a  standard  set  of  descriptive 
statistics:  bias,  threat  score,  and  percent  correct.  Bias  is  defined  here  as  the 
fraction  (F/O)  where  F  and  O  are  the  number  of  forecasts  and  observations  of  the 
same  category,  respectively.  Unit  bias  is  attained  when  F  and  O  are  equal.  Threat 
score  is  defined  as  |Hj/(F+  O-Hj)]  where  Hj  is  the  number  of  correct  forecasts 
of  a  particular  category  (for  example,  ceiling  less  than  1000  ft),  while  F  and  O  are 
as  defined  above.  Percent  correct  is  defined  as  (Hj  +  H^I/TOT  where  (Hj  +  Hg)  is 
the  total  number  of  correct  forecasts  and  TOT  is  the  sample  size.  The  data  in 
Table  6  can  be  used  to  describe  the  method  of  calculating  the  statistics  used  here. 
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Table  (i.  Prediction  and  Observation  Data  I  sed  to  Calculate  Skill 
Measures  for  Categorical  Forecasts 


Forecast  j 

Observed 

Ceiling  <  1000  ft 

Ceiling  >  1000  ft 
or  No  Ceiling 

Total 

Ceiling  <  1000  ft 

2665 

227 

2892 

Ceiling  >  1000  ft  j 
or  No  Ceiling 

216 

2589 

2805 

Total 

2881 

2816 

5697 

For  the  category  ceiling  <  1000  ft,  descriptive  statistics  are 


Bias  -  (2  o2/ 288 1 )  -  1. 004 

Threat  Score  =  '8651/(2892  +  2881  -  2665)  =  0.  857 

Percent  Correct  -  ”00(2665  «■  2589)/5697  =  92.  2 


6.  COMPARATIVE  ANALYSIS 

The  probability  and  categorical  forecast  techniques  were  applied  to  each  epi¬ 
sode  in  the  independent  data  set.  We  combined  the  results  of  each  episode  to  deter¬ 
mine  the  overall  accuracy  and  reliability  of  each  forecast  technique. 

6.1  Probability  Forecasts 

In  our  examination  of  forecast  techniques,  we  wanted  to  demonstrate  the 
feasibility  of  applying  finely  tuned  statistical  methods  rather  than  standard  routines 
(for  example,  climatology  and  persistence).  Accordingly,  we  designed  two 
scenarios.  First,  we  wanted  to  determine  the  additive  contribution  of  visibility  and 
wind  direction  information  (REEP11  and  KEEP.  01)  to  prediction  accuracy  beyond 
using  the  initial  cloud  observation  (KEEP5  and  Equivalent  Markov)  as  a  predictor. 
The  five  predictand  categories  shown  in  Table  1  were  used  in  this  first  scenario. 
Second,  we  wanted  to  compare  the  REEP  and  climatology  techniques.  Since  the 
climatology  tables  did  not  provide  all  five  predictand  categories  listed  in  Table  1, 
a  one-category  predictand  (ceiling  less  than  1000  ft)  was  selected  as  a  means  to 
evaluate  these  techniques.  In  both  scenarios,  persistence  was  used  as  the  control 
technique.  The  techniques  were  compared  by  use  of  the  Brier  Score  (p)  statistic 
which  calculates  the  mean  squared  probability  error  and  by  reliability  graphs. 


for  the  five-predictand  scenario,  verification  results  of  the  p-score  in  Fig¬ 
ure  4,  show  that  KEEP  11  yields  a  slight  improvement  over  KEEP.  01,  REEP5  and 
Equivalent  Markov  as  well  as  a  significant  improvement  over  persistence.  The 
5  percent  maximum  improvement  of  HEEP  11  over  Equivalent  Markov,  when  both 
are  compared  with  persistence  (Table  7),  suggests  that  the  additive  contribution  of 
visibility  and  wind  direction  to  prediction  accuracy  is  small.  This,  in  turn,  sug¬ 
gests  one  could  use  Equivalent  Markov  in  lieu  of  REEP11  with  a  slight  decrease  in 
accuracy.  A  single  equation  set  could  then  be  used  for  all  forecast  intervals.  How¬ 
ever,  an  additional  penalty  is  incurred  because  of  the  poor  resolution  of  the  Equiva¬ 
lent  Markov  model.  This  can  be  seen  in  Figures  5  and  6  for  category  3,  ceiling 
between  500  and  1000  ft.  Note  that  the  reliability  of  both  models  is  very  good. 
However,  Equivalent  Markov  generates  no  forecasts  in  the  0-15  percent  range  while 
23  percent  of  the  REEP11  forecasts  lie  in  this  range.  This  poor  resolution  of 
Equivalent  Markov  holds  for  all  predictand  categories,  and  therefore,  results  in 
part,  in  higher  (less  accurate)  p-scores.  We  therefore  conclude  that  KEEP11  is 
the  preferred  forecast  technique. 
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Figure  4,  Brier  Scores  for  All  Forecasts  Using  a 
Five-Category  Predictand 
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Table  7.  Percent  Improvement  Over  Persistence  in 
Five-Category  Brier  Score 


CATEGORY  3 
EQUIVALENT  MARKOV 
180  MIN  FORECAST 


Figure  5.  Reliability 
Graph  of  Forecasts 
From  the  Equivalent 
Markov  Model  for 
180  min.  Ceiling 
500-1000  ft 
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CATEGORY  3 
REEP1 1 

180  MIN  FORECAST 
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Figure  6.  Reliability 
Graph  of  Forecasts 
From  the  REEP11 
Model  for  180  min. 
Ceiling  500-1000  ft 
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For  the  one-predictand  scenario,  categorical  probabilities  were  converted  to 
exceedance  probabilities.  This  was  simply  done  after  the  (Pj,  P,,,  Pg,  P^,  P,.) 
were  known.  For  example,  the  probability  a  ceiling  will  be  below  1000  ft  (Pg  or 
less)  given  (0.  2,  0.  5,  0.  1,  0.  15,  0.  05)  is 

P(ceiling  <  1000  ft)  =  P^  +  P^  +  Pg  • 

Hence 

Piceiling  <  1000  ft)=  0.8  . 

Verification  results  of  the  p-score  are  shown  in  Figure  7.  They  show  that  REEPll 
yields  a  significant  improvement  over  Equivalent  Markov,  climatology,  and  per¬ 
sistence.  Also,  REEP11  yields  marginal  improvement  over  KEEP.  01  and  REEP5. 
The  improvement  of  REEPll  over  Equivalent  Markov  is  as  much  as  10  percentage 
points  (see  Table  8),  with  the  underlying  reason  the  poor  forecast  resolution  of  the 
Equivalent  Markov  model.  Figures  8  and  9  show  that  while  REEPll  makes  approxi¬ 
mately  46  percent  of  its  forecasts  in  the  (0  to  5)  and  (85  to  100)  percent  ranges. 
Equivalent  Markov  does  not  make  any.  Figure  10  shows  that  the  poor  performance 
of  climatology  is  due  to  the  systematic  bias  to  underforecast  and  its  relatively 
poorer  resolution.  Therefore,  as  it  is  in  the  five-predictand  case,  REEPll  is 
the  most  accurate  and  reliable  forecast  technique  for  a  single  predictand. 
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Figure  7.  Brier  Scores 
for  All  Forecasts  Using 
a  One-Category  Predictand 
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Table  8.  Percent  Improvement  Over  Persistence  in 
One-Category  Brier  Score 


Prediction 

Technique 

Forecast  Length  (min) 

30  60  120  180 

KEEP 1 1 

14.  1 

21.3 

28.  6 

33.  5 

KEEP.  01 

12.  8 

20.  5 

28.  1 

33.  1 

KEEP5 

1 0.  3 

14.  8 

21.9 

27.  8 

Equivalent  Markov 

10.  3 

13.  1 

18.  8 

23.  8 

Climatology 

— 

4.  1 

13.  6 

20.  6 
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Figure  8.  He  liability 
Graph  of  Forecasts 
From  the  HEEPll 
Model  for  180  min, 
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6.2  Categorical  Forecasts 

The  two  "category  predictand  enabled  us  to  compare  various  techniques  for  con¬ 
verting  a  probability  estimate  into  a  categorical  forecast.  Three  significant  results 
can  be  drawn  from  Tables  0  through  12.  categorical  forecast  statistical  summaries. 
One  objective  was  to  compare  converting  KEEP  and  climatology  probabilities  into 
categorical  forecasts  using  the  maximum  probability  model,  Max  Pr  >b  K  and 
Max  Prob  c,  respectively.  Here,  REEP  consistently  yielded  marginally  better  re¬ 
sults  than  climatology.  Also,  REEP  systematically  overcast  the  event  while 
climatology  underforecast  it.  The  second  objective  was  to  assess  the  relative  skill 
of  the  iterative  method  and  the  comparatively  simple  unit  bias  method.  The  unit 

bias  method  yields  results  superior  to  the  iterative  method  coinciding  with  findings 

12  * 
made  by  TDL.  In  effect,  one  can  simply  calculate  p  by  having  the  (R,e)  pair 

rather  than  having  to  further  utilize  the  developmental  sample.  Finally  and  most 

important,  persistence  forecast  results  are  extremely  competitive  with  the  other 

methods.  Note  that  persistence  forecast  the  event  as  often  as  it  occurs  and  vielded 

similar  threat  and  percent  correct  statistics. 


Table  9.  Descriptive  Statistics  lor  30-min 
Categorical  Forecasts 


- tl 


Method 

Bias 

Threat 

Score 

Fercent 

Correct 

Max  Frob  K 

1.  004 

0.  H57 

90.  6 

Fat  -dstence 

0.  857 

92.  > 

Iterative 

0.  90S 

0.7  10 

8  '.  2 

Unit  Bias 

0.  857 

90.  6 

Table  10.  Descriptive  Statistics  for  60-min 
Categorical  Forecasts 


Method 

Bias 

Threat 

Score 

Percent 

Correct 

Max  Prob  R 

1.  002 

0.  786 

87.  8 

Max  Prob  o 

0.  97  5 

0.766 

86.7 

Persistence 

1.  002 

0.786 

87. 8 

Iterative 

0.  900 

0.731 

85.  9 

Unit  Bias 

1.  002 

0.786 

87. 8 

Table  11.  Descriptive  Statistics  for  120-min 
Categorical  Forecasts 


Method 

Bias 

Threat 

Score 

Percent 

Correct 

Max  Prob  R 

1.090 

0.  696 

79.  9 

Max  Prob  c 

0.  966 

0.  642 

78.0 

Persistence 

0.  96? 

0.  665 

80.7 

Iterative 

0.927 

0.  67  5 

80.  6 

Unit  Bias 

I.  209 

0.  696 

79.  7 

Table  12.  Descriptive  Statistics  for  180-min 
Categorical  Forecasts 


Method 

Bias 

Threat 

Score 

Percent 

Correct 

Max  Prob  R 

1.  062 

0.  627 

75.  6 

Max  Prob  c 

0.  926 

0.  567 

72.  5 

Persistence 

1.  003 

0.  613 

7  5.  2 

Iterative 

0.  900 

D.  595 

75.  2 

Unit  Bias 

1.  156 

0.  661 

77.3 

7.  SUMMARY  AND  CONCLUSIONS 

Analysis  of  the  data  collected  at  the  APGL  Weather  Test  Facility  demonstrated 
the  accuracy  and  reliability  of  finely  tuned  statistical  methods  for  low  cloud  ceiling 
prediction.  In  an  examination  of  probability  forecasts,  REEP  was  found  to  yield 
significantly  better  results  on  independent  data  than  did  the  Equivalent  Markov  tech¬ 
nique,  climatology,  and  persistence.  In  addition,  visibility  and  wind  direction 
information  yielded  marginal  improvement  to  the  accuracy  and  reliability  of  the 
REEP  probability  forecasts  beyond  that  already  attained  using  only  the  initial  cloud 
observation  as  a  predictor.  Specifically,  REEP11  had  the  greatest  accuracy  and 
superior  forecast  resolution. 

In  making  categorical  forecasts,  persistence  yielded  extremely  competitive 
results  on  independent  data  when  compar  e  with  the  thresholding  methods. 

It  is  concluded  that  these  techniques  provide  timely,  cost-effective  decision 
aids  to  the  local  decision  maker.  Therefore,  they  satisfy,  in  part,  AWS  require¬ 
ments  in  modernizing  its  basic  weather  support  capabilities.  The  development  and 
examination  of  these  forecast  techniques  is  a  part  of  AFGL's  continuing  investiga¬ 
tion  of  automated  airfield  weather  observation  and  prediction  systems. 
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Appendix  A 

Coefficient  Sets  for  REEP  Forecast  Equations 


1.  REEP5  Prediction  Equation  Set 


30-min  Forecast 

Order(i) 

P ! 

P2 

F  P 

3  4 

P5 

Predictor 

X. 

i 

Total 

Reduction 

of  Variance 

0.  539 

0.  518 

0.  478 

0.  357 

0.  564 

31 


.  124 

0.  679 

0.  180 

0 

.  098 

-0.  538 

0.  641 

5 

.  124 

-0.  679 

-0.  099 

1 

.  034 

-0.  664 

-0.  169 

2 

.  638 

-0.  584 

-0.  141 

3 

0 

0 

0.  009 

1 

0 

0.  003 

2 

0.  727 

0.  183 

3 

0.  055 

0.  752 

4 

0 

0.  095 

OTOlfiDO  Pifii  aUMUMOT  FILMS 


KEEPS  Prediction  Equation  Set  (Cont) 


Order(i) 


Reduction 
of  Variance 


60-min  Forecast 


Predictor 

X. 


0 

0 

0.  188 

0.  632 

0 

108 

0 

.  066 

0 

1 

0.  005 

-0.  166 

-0. 600 

0 

.  085 

0 

.  682 

5 

2 

0.  626 

0.  025 

-0.582 

0 

.  078 

0 

.  015 

1 

3 

0.  081 

0.  44  1 

-0.408 

0 

.  080 

0 

.  028 

2 

4 

0 

-0.  167 

-0.  457 

0 

.  491 

0 

.  139 

4 

120-min  Forecast 


Order(i) 


Total 
Reduction 
of  Variance 


.  009 

0.  247 

.  547 

0.  005 

- 

.  008 

-0.  204 

- 

.  006 

-0.  212 

- 

.  090 

0.  234 

- 

.  506 

.475 

- 

.443 

.  249 

.  220 

- 

.  130 

.  100 

.  110 

.  300 

.  062 

- 

P_ 

D 

Predictor 

X. 

l 

).  102 

0 

).  029 

1 

).  551 

5 

).  172 

4 

).  036 

2 

0.  162  0.  099 


180-min 

Forecast 

Order(i) 

P1 

P2 

P3 

P4 

P5 

Predictor 

X. 

l 

0 

0.  026 

0.  232 

0.  451 

0.  125 

0.  159 

0 

1 

0.418 

0 

-0. 380 

-0.  024 

-0.  007 

1 

2 

-0.  024 

-0.  166 

-0.  356 

0.  176 

0.  376 

5 

3 

0.  086 

0.  201 

-0.  203 

-0.  038 

-0.  040 

2 

4 

-0.  022 

-0.  162 

-0.  188 

0.  229 

0.  149 

4 

Total 
Reduction 
of  Variance 


0.107  0.067 


KKKF.O  I  Prediction  rxj  nation  Set 


30-min 

Forecast 

Order(i) 

P1 

P2 

P3 

P4 

P5 

Predictor 

X. 

i 

0 

0 

-0.  001 

0.  092 

0.695 

0.  207 

0 

1 

0 

0.  008 

-0.  080 

-0.  546 

0.  626 

5 

2 

0.  523 

-0.  052 

0.  055 

-0.  590 

0.  070 

1 

3 

-0.  008 

0.  526 

0.  115 

-0.  585 

-0.  043 

2 

4 

-0.  002 

0.  046 

0.  585 

-0.  546 

-0.  077 

3 

5 

0.  257 

0.  242 

-0. 179 

-0.  107 

-0.  211 

9 

6 

0.  026 

0.  340 

-0.  100 

-0.  107 

-0.  157 

8 

7 

-0.  001 

0.  044 

0.  163 

-0.  074 

-0.  130 

7 

Total 
Reduction 
of  Variance 

0.  594 

0.  559 

0.  512 

0.  358 

0.  575 

60-min  Forecast 

Order(i) 

P1 

P2 

P3 

P4 

P5 

Predictor 

X. 

l 

Total 
Heduction 
of  Variance 


.001 

0.  180 

0.695 

0.  096 

0.  022 

0 

.  012 

-0.  099 

-0.  549 

0.  041 

0.  599 

5 

.406 

-0.  227 

-0.  323 

0.024 

0.  126 

1 

.  012 

0.  247 

-0.  266 

-0.  013 

0.  025 

2 

.  006 

-0.  108 

-0.420 

0.454 

0.  070 

4 

.  278 

0.  27 3 

-0.  360 

-0.  100 

-0.  088 

9 

.  033 

0.  291 

-0.  248 

-0.  073 

0 

8 

.  006 

-0.  066 

-0.  125 

0.  063 

0.  139 

6 

.441 

0.  364 

0.  329 

0.  245 

0.445 

REEP.OI  Prediction  Equation  Set  (Coni) 


120-min 

Forecast 

Order(i) 

F1 

P2 

P3 

H4 

P5 

Predictor 

X. 

l 

0 

0 

0.  179 

0.  384 

0.  218 

0.  207 

0 

1 

0.  322 

0.  290 

-0.  188 

-0.  133 

-0.  280 

9 

2 

-0.  001 

-0.  145 

-0.  337 

0.  034 

0.  460 

5 

:i 

0.  048 

0.  345 

-0.  101 

-0.  158 

-0.  123 

8 

4 

0.  296 

-0.  213 

-0.  201 

-0.  052 

0.  169 

1 

5 

0 

-0.  165 

-0.  163 

0.  240 

0.  098 

4 

6 

0.  014 

0.  100 

0.  176 

-0.  134 

0.  147 

7 

Total 
Keduction 
of  Variance 

0.  358 

0.  202 

0.  :  85 

0.  1 12 

0.  294 

180-min 

Forecast 

Order(i) 

P1 

P2 

'.3 

P4 

0 

Predictor 

X. 

i 

0 

0.  063 

0.  250 

0.  322 

0.  216 

0.  138 

0 

1 

0.  277 

-0.  148 

0.  020 

0.  042 

-0.  189 

9 

2 

-0.  036 

-0.  150 

-0.  257 

0.  120 

0.  333 

5 

3 

0.  028 

0.  247 

-0.  139 

-0.  081 

-0.  046 

2 

4 

-0.  009 

0.  059 

0.  202 

-0.  100 

-0.  145 

7 

5 

-0.  035 

-0.  156 

-0.  115 

0.  183 

0.  132 

4 

6 

-0.  058 

-0.  102 

-0.  041 

-0.  041 

0.  247 

13 

7 

0.  187 

0.  129 

-0.  269 

-0.  127 

0.  088 

1 

8 

-0.  048 

-0.  076 

0.  054 

-0.  061 

0.  134 

10 

Total 
Reduction 
of  Variance 

0.  258 

0.  143 

0.  144 

0.  080 

0.  195 

3.  RK.KPl  1  Prwl 

ict ion  F|uation 

Sets 

30-min 

Forecast 

Order(i) 

P1 

‘*2 

P3 

P4 

l\ 

0 

Predictor 

X. 

l 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0.  009 

-0.  083 

-0.  550 

0.  623 

5 

•> 

0.  523 

-0.  045 

0.  044 

-0.  584 

0.  06  2 

1 

3 

-0.  004 

0.  527 

0.  106 

-0.  582 

-0.  047 

2 

4 

0.  001 

0.  05  1 

0.  574 

-0.  542 

-0.  084 

3 

5 

0.  254 

0.  278 

-0.  106 

0.  598 

-0.  023 

u 

6 

0.  025 

0.  373 

-0.  028 

(  .  599 

0.  03  i 

8 

7 

-0.  002 

0.  07  5 

0.  235 

0.  633 

0.  059 

7 

a 

0 

0.  031 

0.  070 

0.  710 

0.  189 

6 

g 

-0.  008 

-0.  065 

0.  072 

-0.  050 

0.  051 

13 

10 

0.  009 

-0.  039 

0.  013 

-0.  002 

0.  019 

12 

11 

-0.  008 

-0.  031 

0.030 

-0.  012 

0.  022 

10 

Total 
Reduction 
of  Variance 


0.  515  0.364 


60-min  Forecast 


Order(i) 

P1 

P2 

P3 

P4 

P5 

Predictor 

X. 

i 

0 

0 

0 

0 

0 

0 

0 

1 

0.  008 

-0.  106 

-0.  540 

0.  031 

0.  607 

5 

2 

0.401 

-0.  219 

-0.  331 

0.  018 

0.  130 

1 

3 

0.  013 

0.  241 

-0.  271 

-0.  010 

0.  028 

2 

4 

0.  002 

-0.  118 

-0.  408 

0.442 

0.  082 

4 

5 

0.  264 

0.477 

0.  341 

-0.  025 

-0.  058 

9 

6 

0.  022 

0.  491 

0.448 

0.  008 

0.  031 

8 

7 

-0.  015 

0.  134 

0.  561 

0.  151 

0.  170 

6 

8 

0.  003 

-0.  074 

0.  069 

-0.  180 

0.  021 

13 

9 

-0. 011 

0.  200 

0.  694 

0.  085 

0.  033 

7 

10 

0.  025 

-0.  019 

-0.028 

0.  049 

-0.  028 

12 

11 

0.  012 

0.  025 

0.  006 

0.010 

-0.  053 

11 

Total 
Reduction 
of  Variance 

0.  445 

0.  371 

0.  339 

0.  249 

0.  448 

3.  REEP11  Prediction  Equation  Sets  (Cont) 


120'min  Forecast 

Order(i) 

P1 

P2 

P3 

P4 

P5 

Predictor 

X. 

l 

0 

0 

0 

0 

0 

0 

0 

1 

0.  301 

0.  198 

-0.  065 

-0.  128 

-0.  307 

9 

2 

-0.  025 

-0.  212 

0.  046 

0.  441 

5 

3 

0.  033 

-0.  012 

-0.  150 

-0.  144 

8 

4 

0.  287 

-0.  220 

-0.  17  1 

-0.  058 

0.  162 

1 

5 

-0.  025 

-0.  273 

-0.  03  i 

0.  251 

0.  084 

4 

6 

0.  011 

0.  091 

0.  183 

-0.  138 

-0.  147 

7 

7 

0.  003 

0.  206 

0.  273 

0.  197 

0.  322 

13 

8 

0.  007 

0.  282 

0.  272 

0.  191 

0.  248 

10 

9 

-0.  020 

in 

o 

© 

1 

0.  141 

0.  023 

-0.  028 

3 

10 

0.  040 

0.  333 

0.  291 

0.  185 

0.  151 

11 

11 

0.  042 

0.  294 

0.  223 

0.  236 

0.  206 

12 

Total 
Reduction 
of  Variance 

0.  368 

0.  217 

0.  195 

0.  115 

0.  307 

180-min  Forecast 

Order(i) 

P1 

P2 

P3 

P4 

P5 

Predictor 

X. 

i 

0 

0 

0 

0 

0 

0 

0 

1 

0.  361 

0.  167 

0.  400 

0.  190 

-0.  118 

9 

2 

-0.  280 

-0.  121 

-0.  233 

0.  082 

0.  299 

5 

3 

0.  008 

0.  177 

-0.  201 

-0.  006 

0.  022 

2 

4 

0.  056 

0.  302 

0.  519 

0.  127 

-0.  004 

7 

5 

-0.027 

-0.  130 

-0.  094 

0.  150 

0.  102 

4 

6 

-0.  060 

-0.  092 

-0.033 

-0.  055 

0.  240 

13 

7 

0.  166 

0.  058 

-0.  332 

-0.  051 

0.  159 

1 

8 

-0.  050 

-0.  069 

0.  060 

-0.  072 

0.  131 

10 

9 

0.  096 

0.  350 

0.409 

0.  Ill 

0.  034 

8 

10 

0.  055 

0.  209 

0.  287 

0.  269 

0.  180 

6 

11 

-0.  077 

0.  026 

0.  024 

-0.  032 

-0.  010 

11 

Total 

Reduction 

of  Variance  0.  260 

0.  150 

0.  149 

0.  087 

0.  200 
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Appendix  B 

Transition  Matrix 

The  following  transition  matrix  was  user)  in  the  Equivalent  Markov  prediction 
technique  and  r  an  he  interpreted  in  the  following  wav; 


)>  p  p 

11  12  15 


p  p 
2  1  >2 


P-,  P-. 

o  1  r> j 


P25 


P55 


where 


P  PICE1L1NG, 
i,l  1  t 


CEILING 


0 


j]  tor  i,  j,  -1,2 . 5 


That  is,  P. ,  is  the  transition  or  conditional  probability  that  the  ceiling  category  at 
time  t.  will  be  in  state  i  given  the  ceiling  category  at  time  0  was  in  state  j. 

30-min  One-Step  Transition  Matrix 


0.7  27 

0.  192 

0. 

000 

0.  000 

0.  081 

0.  055 

0.7  60 

0. 

158 

0.  015 

0.  011 

0.  000 

0.  104 

0. 

762 

0.  095 

0.  039 

0.000 

0.  009 

0. 

125 

0.  684 

0.  181 

0.000 

0.011 

0. 

026 

0.  141 

0.  821 
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Appendix  C 

Coefficient  Set  for  REEP  Equation 
Categorical  Forecasts 


The  following  set  of  REEP  equations  was  used  in  the  iterative  and  unit  bias 
thresholding  models  (see  Section  4).  Each  forecast  equation  generates  a  probability 
estimate  (P)  of  a  ceiling  less  than  1000  ft.  See  Table  4  for  a  summary  of  predictor 
limits.  For  example,  what  is  the  probability  a  ceiling  less  than  1000  ft  will  occur 
in  30  min  given  the  observation  matrix  O  =  ( 1,  0,  0,  0,  0,  0,  0,  0,  1.  0,  1,  0,  0,  )? 

P  -  P(CEILING  <  1000  ft)  0.425  -  0.  523  =  0.948  . 

Also, 

P(CEILING  2  1000  ft  or  no  ceiling)  =  1.0-P  =  0.052  , 
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1.  REEP  Coefficients  Estimate  for  Ceiling  Less  Than  1000  ft 


Forecast  L 

ength  (min) 

30 

60 

Order(i) 

P 

Predictor 

X(i> 

P 

Predictor 

-\(i) 

0 

0.  425 

0 

0.  942 

0 

1 

-0. 324 

6 

-0.  282 

6 

2 

0.626 

3 

-0.  620 

5 

3 

0.  629 

2 

-0.  506 

4 

4 

0.  523 

1 

-0.  07  8 

7 

5 

-0.  07  3 

5 

0.  044 

1 1 

6 

-0.  118 

7 

-0.  13  1 

i 

7 

-0.  055 

8 

0.  122 

9 

8 

-0.  017 

12 

-0.  020 

12 

9 

-0.  010 

10 

0.  018 

3 

Total 

Reduction 

of 

Variance 

0.  850 

0.  778 

Forecast  Length  (min) 

120 

180 

Order(i) 

P 

Predictor 

XU) 

P 

Predictor 

X(i) 

0 

0.  857 

0 

0.  87  5 

0 

1 

-0.  284 

6 

-0.  325 

6 

2 

-0.  500 

5 

-0. 379 

5 

3 

-0.  348 

4 

-0.  249 

4 

4 

0.  104 

1 1 

-0.  185 

13 

5 

-0.  07  8 

13 

-0.  059 

10 

6 

0.  138 

9 

0.  041 

11 

7 

-0.  106 

1 

-0.  094 

1 

8 

-0.  031 

8 

.9 

0.  038 

9 

Total 

Reduction 

of 

Variance 

0. 

679 

0. 

594 
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